Contemporary Issues In Education Research - June 2011 


Volume 4, Number 6 


A Multicenter Study Of Students ’ 
Sensitivity To Screen-Update Delay 

Vonda K. Walsh, Virginia Military Institute, USA 
H. Francis Bush, Virginia Military Institute, USA 
lames C. Squire, Virginia Military Institute, USA 
Gerald A. Sullivan, Virginia Military Institute, USA 


ABSTRACT 

In a climate of shrinking educational budgets, online learning courses offer many advantages; 
however there are several possible problems associated with electronic learning. There may be 
problems associated with learning style preferences, student apathy, instructional support, 
accessibility, and problems with technology. This paper examines a technological problem 
associated with the effects of screen-update-delay and student enjoyment, student self-reported 
comprehension, and student objective comprehension. In particular, this paper attempts to 
identify the point at which longer screen updates will be detrimental to the student’s enjoyment 
and/or have a significant negative impact on the student’s self-reported comprehension and 
retention of material. 

Keywords: multicenter study; online learning; online courses; screen-update delay; comprehension; self-reported 
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INTRODUCTION 



inline education continues to grow with no signs of slowing down. During the 2006-2007 academic years, 
'61% of 2-year and 4-year educational institutions were offering online courses (Parsad & Lewis, 2008, 
pi). According to Sloan Consortium research, “over 3.9 million students were taking an online course 
during the fall 2007 term” (Allen & Seaman, 2008, p. 1). As colleges and universities become more comfortable 
with online classes and as they are pressed to cut costs, more colleges and universities are increasing the number of 
online courses they are mixing with their traditional campus-based classes. Many administrators believe that their 
costs will be substantially reduced as they implement more online courses; they may also believe they will reach a 
wider audience. 


Some of the biggest advantages with online learning are an increased level of student engagement and 
participation and the immediacy of appropriate feedback to the student. It is important to keep learners motivated 
and attentive so that they will continue with their work. Since all individuals learn best by personally interacting 
with the material, effective teaching practices should engage the student, encourage the development of learning 
skills, and motivate additional learning. Students need to be engaged, need to be given feedback, and after they have 
demonstrated proficiency in the presented material, should be encouraged to proceed to the next learning scenario. 

The constructivist model of learning is based upon the principle that students learn best by actively 
constructing their own ways of organizing information for effective understanding and recalling of material 
(Lockwood, 2001). One of the primary assumptions under this model is that techniques for organizing information 
for retention are in constant development. If the development of these cognitive structures is interrupted, then the 
student may fail to learn and may not experience the enjoyment of learning. So as with traditional methods of 
instructions, computer or web-based instructions must engage the student, must give adequate feedback, and must 
provide the student with ample opportunity to organize new information for retention. 
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While computers can engage students and provide rapid feedback, the effective delivery of instructional 
material to multiple network users may be interrupted by screen-update delay. Web-based simulations allow the 
student to evaluate and compare various consequences of multiple inputs, and these simulation models can act as 
templates to integrate various components so that the student can develop an understanding of the interactions of 
these components. Furthermore, web-based simulation models can provide students with opportunities to construct 
or develop their own understanding of the material. The critical point is to keep the student motivated and engaged 
in learning. If there is an interruption of the interaction of the student with the web-based models, the student may 
become disengaged with the learning process. 

Additionally, students’ expectations of success or failure may impact their approach to and their enjoyment 
of the learning scenario and their objective comprehension of the scenario. Ehrlinger and Dunning (2003) opined 
that this may have to do with students’ chronic self-views about their abilities and the perceived notions about 
whether they are skilled or unskilled at a particular task, but Heine (1999) showed that their inflated self-assessment 
is more common in Western cultures. Earlier research has indicated that an increase in screen-update delay impacts 
the learning experience when measuring objective learning, enjoyment, and self-reported comprehension (Squire, et 
al, 2008); (Bush, et al, 2008). 

This paper will explore in greater detail the effect of screen-update delay on the student’s enjoyment, the 
student’s objective learning, and the student’s reported self-confidence of the learning experience. More specifically, 
we will test the following three hypotheses: 

1. Student-reported level of enjoyment will decline as the length of the screen-update delay increases. 

2. Student-objective score will decline as the length of the screen-update delay increases. 

3. Student-reported level of self-confidence will decline as the length of the screen-update delay increases. 

We employed a simulation model designed to reinforce concepts in Fourier Analysis. The simulation 
model was embedded with different levels of screen-update delay between the time a subject manipulated the 
controls and the appearance of the update on the computer screen. Scores for the students’ objective comprehension 
and their perceptions of their enjoyment and of their own comprehension were obtained. 

METHODS 

The age range of participants was 15-25 years of age, with a mean age of 19.15 years. Ninety-six students 
identified their major as "humanities”, 155 students identified their major as engineering or sciences, and 30 did not 
identify a single major of study. Males were more represented (86.1%) than females. This overrepresentation of 
males was probably due to the majority of the students coming from engineering or sciences. While there was a 
wide range of skill level using the technology, all students were comfortable with using computers and surfing the 
internet. 


The study involved 281 students from four different universities over a broad range of disciplines. While 
the study was a double blind experiment, participants were recruited in classroom settings using formal 
presentations describing the procedures and risks. The subjects were told that they were testing an instructional 
program about Fourier Analysis concepts. The actual goal of the research was not revealed to participants and 
authorization was obtained from the Human Subjects and Animal Use Committee at Virginia Military Institute. 

A C# interactive software application, containing a hidden embedded screen-update-delay between the time 
the subject manipulated the controls and the time the screen was updated, was designed. Eight different versions of 
instructional application were designed with the only difference being the delay (Squire, et al, 2008). A screenshot 
of the application is shown in Figure 1 and can be downloaded at 
http://www2.vmi.edu/Faculty/squirejc/Research/Fourier Synthesis/Fourier Synthesis.htm . 

The 281 students were randomly assigned to one of the eight evenly-spaced time interval applications 
ranging from 0 to 420 ms. 1 Each student completed a self-guided learning tutorial which began by collecting some 
demographic information. Throughout the tutorial, students were directed and encouraged to experiment with 
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various sliders within the computer-based tutorial to visually see the results of their interactions with the tutorial. 
The tutorial consisted of “read, apply and respond”-type instructions. Cognitive feedback was involved in all 
interactions with the tutorial and students were able to observe the consequences of their own actions. The objective 
comprehension score was the subject’s total of correct responses on 15 multiple choice questions that required the 
students to use the Fourier Analysis program. The final two questions ask the students to self-report how much they 
enjoyed the tutorial and how much they thought they learned about Fourier Analysis. Five-point Likert Scales were 
used to assess student’s self-reported enjoyment and student’s self-reported comprehension. 

A Matlab program analyzed the data, scored the objective questions, and then plotted error bars indicating 
one standard deviation above the mean for each of the eight latency groups. Finally, the application graphed the 
best-fit horizontal, linear and bilinear lines. These graphs are derived by forming sequential linear best fit lines. 
The graphs are produced by going through every data point, bending and twisting as needed to generate the fewest 
number of line segments (in this case two) connecting all the data points. Based on the correlation coefficient, we 
conclude that a linear split function with two components does the best job at capturing the trend of the data. 



Figure 1: Fourier Synthesis Application Program 


RESULTS 

Results from the experiment are shown in Figures 2 through 4. These results represent the relationship 
between screen-update delay and self-reported enjoyment, objective learning, and self-reported comprehension. 
Each figure shows error bars with one standard deviation from the mean with the best-fit bilinear line superimposed 
on the data histogram. Two types of bilinear lines were computed; one started with a horizontal segment and the 
other ended with a horizontal segment. The one with the smallest residuals was selected for analysis. 

In Figure 2 (Latency vs. enjoyment), the curve is similar to a failure rate curve; i.e., the bathtub curve. The 
initial region (0-300 ms) shows the greatest rate of decline with a slope of -0.19089. This slope is significant with a 
p-value < 0.001. This supports our first hypothesis that student-reported level of enjoyment will decline as the 
length of screen-update delay increases. The median enjoyment score is 4.2 and occurs at time = (300+0)/2= 150 
ms. The initial region is where failure is the greatest; i.e., students’ enjoyment will decrease the fastest in the (0- 
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300ms) range. This area is challenging since it may be difficult to maintain a screen-update delay less than 300 ms 
and thus difficult to maintain student enjoyment. Students may disengage and not participate further in the learning 
module. The next region (300-420 ms) has a relatively constant failure rate. 11 We would like for this approximately 
horizontal line to be as high as possible. In this region, we have an enjoyment score of 3.7. So even when screen- 
update delay is greater than 300 ms, the student’s enjoyment does not decrease. These results are consistent with 
our first hypothesis and indicate the need to apply resources to decrease screen-update delay to increase user’s 
enjoyment. 



Figure 2: The Relationship between Student-reported Levels of Enjoyment of An Interactive 
Teaching Software Application and Screen-Update Delay 


Statistics: 

First Line 

Estimated slope = -0.19089 
Ho: (3 = 0 

t-value = -5.60926; n = 217 
p-value = 6.21 E -8 


Second Line 

Estimated slope = 0.010528 
Ho: (3 = 0 

t-value = 0.087655; n = 95 
p-value = 0.930339 


In Figure 3 (Latency vs. objective learning), again we have a bathtub curve. The initial region (0-60 ms) 
shows the greatest rate of decline with a median objective learning score of 73%. The slope in the initial region is 
-0.07073 with a p-value < 0.0589. 111 The constant failure rate region (60-420 ms) has an objective learning score of 
69%. This is probably due to the fact that students are not given ample opportunity to develop their own 
understanding of the material. Again, it is desirable to have the steady state horizontal portion of the graph to be as 
high as possible. 1V 

In Figure 4 (Latency vs. self-reported comprehension), the graph is different from the previous two graphs; 
it is an inverted bathtub. It begins with a steady state region (0 - 300ms) with a median score of 2.6. This steady 
state horizontal portion (constant failure rate) v of the graph seems to indicate that screen-update delays from 0 to 300 
ms are well tolerated by students when compared to students’ self-reported comprehension (i.e., the student 
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maintains a high level of self-confidence within this range of delay). The slope of the second portion of the graph 
(300-420 ms) is -0.2884 and is slope is significant with a p-value = 0.019. This supports our third hypothesis that 
students’ self-reported comprehension will decline as the length of screen-update delay increases. Once the screen- 
update delays reach this 300 ms range, students may begin to have significant doubt or anxiety regarding their 
comprehension. 



Figure 3: The Relationship between Objectively-scored Measures of Student Comprehension 
and of an Interactive Teaching Software Application and Screen-update Delay 


Statistics: 

First Line 

Estimated slope = -0.07073 
Ho: (3 = 0 

t-value =-1.91671; n = 81 
p-value = 0.058891 


Second Line 

Estimated slope = 0.001883 
Ho: (3 = 0 

t-value = 0.374087; n = 240 
p-value = 0.708673 


CONCLUSIONS AND FUTURE CONSIDERATIONS 

Results demonstrate that students are sensitive to screen-update delays and clearly indicate that student- 
reported level of enjoyment, objective scores, and self-reported comprehension all decrease as the length of screen- 
update delay increases. 

Comparing the graphs for enjoyment, objective learning, and self-reported comprehension shows that the 
rate of decline is greatest for enjoyment. One reason may be the simplicity of the self-guided tutorial. The tutorial 
was fairly easy to implement and to understand. It is also possible that the students perceived their ability as an 
incremental skill that is continually developing. The students were probably looking for their personal best 
performance and were enjoying the race to perfection rather than manipulating the tutorial. 
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Figure 4: The Relationship between Self-reported Comprehension of an Interactive Teaching 
Software Application and Screen-update Delay 


Statistics: 

First Line 

Estimated slope = -0.07073 
Ho: (3 = 0 


t-value = -0.25723; n = 217 
p-value = 0.797249 


Second Line 

Estimated slope = 0.001883 
Ho: (3 = 0 

t-value = -2.39983; n = 95 
p-value = 0.018395 


A limitation of this research is the overrepresentation of males in the study. The results indicate a clear 
sensitivity to screen-update delays, but future research will be necessary to demonstrate whether this is true across 
genders and academic majors and to see if there are any interactions between gender and academic majors. 

It should also be noted that all three of the curves depict expected behavior of our population of students on 
a relatively simple simulation instructional model and should only be used by others to illustrate possible student 
behavior; however, it is clear that all results indicate that it is appropriate to apply resources to reduce screen-update 
delays and that actual and perceived learning varies inversely with the length of screen-update delay. 
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I A pilot study was performed with 48 test subjects to determine the range of screen-update delays. Based on this study, it was 
decided that 420 ms would be sufficient. 

II When testing the slope ((3 =0) for the second region in Figure 2, we obtain a p-value < 0.940. We do not expect a substantial 
decline in enjoyment when screen-update delays are greater than 300 ms. 

III Although the level of significant (p < 0.120) for the first region of Figure 3 is above the traditional cutoffs, the results are 
consistent with our hypothesis that student reported level of enjoyment will decline as the length of screen-update delay 
increases. A possible cause is the simplicity of the task. 

IV When testing the slope ((3 =0) for the second region in Figure 3, we obtain a p-value < 0.709. We do not expect a substantial 
decline in objective learning when screen-update delays are greater than 60 ms. 

v When testing the slope ((3 =0) for the first region in Figure 4, we obtain a p-value < 0.798. We do not expect a substantial 
decline in self-reported comprehension when screen-update delays are less than 300 ms. 
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